test: fix stale/wrong test assertions and CI trigger gap - #19
Conversation
- test_config_validation.py: KNOWN_TASK_TYPES and the model-registry check only covered timeseries+vision, so every audio modelzoo example (task_type='audio_classification', e.g. DSCNN_NPU) looked invalid. Include the audio module's task types and route audio configs to its own model registry. - test_constants.py: get_default_data_dir_for_task() was intentionally changed to raise ValueError on unknown categories in 90ce979 (CodeRabbit fix), but the test still asserted the old silent DATA_DIR_CLASSES fallback. Update the test to match. - test_cross_device.py: TestQuantizationFlags called get_skip_normalize_and_output_int(..., partial_quantization=...), but the real parameter is auto_quantization -- a wrong kwarg name, not a real API. Rename calls and the one test that was actually about auto_quantization semantics. - test_dataset_utils.py: test_split_factor_too_large matched a stale error-message regex ("less than 1") against the current message ("must be in the range (0.0, 1.0)"); behavior was already correct. Confirmed via git stash that all of the above failed identically before the unrelated ConfigDict deep-merge fix, i.e. pre-existing and unrelated to it. One real gap found and left open per user decision: F28E12 is listed in TARGET_DEVICES but has no entry in _DEVICE_PROFILES, so test_cross_device.py::TestCompilationProfileCorrectness still fails for it (5 tests). Needs real hardware values (cross_compiler path, target, target_c_mcpu, has_hard_npu) that aren't safe to guess. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
tinyml-modelmaker depends on tinyml-modelzoo (pip install -e tinyml-modelzoo in the Install dependencies step), but dependency-only changes to it wouldn't trigger the test suite at all. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
CI failure is the separately-documented
Worth noting this PR's own fix does work as intended: its run shows zero |
# Conflicts: # .github/workflows/test-modelmaker.yml
bdb59f7 to
66e2588
Compare
3 participants
Summary
Two independent test/CI hygiene fixes, confirmed unrelated to any production code change (verified via
git stash— all of the below fail identically on unmodifiedmain).Stale/wrong test assertions
test_config_validation.py:KNOWN_TASK_TYPESand the model-registry check only covered timeseries + vision, so every audio modelzoo example (task_type='audio_classification', e.g.DSCNN_NPU) looked invalid. Includes the audio module's task types and routes audio configs to its own model registry.test_constants.py:get_default_data_dir_for_task()now raisesValueErroron unknown categories, but the test still asserted the old silentDATA_DIR_CLASSESfallback. Updated to match current behavior.test_cross_device.py:TestQuantizationFlagscalledget_skip_normalize_and_output_int(..., partial_quantization=...), but the real parameter isauto_quantization— a wrong kwarg name, not a real API. Renamed calls; the one test that was actually aboutauto_quantizationsemantics is kept meaningful.test_dataset_utils.py:test_split_factor_too_largematched a stale error-message regex ("less than 1") against the current message ("must be in the range (0.0, 1.0)"); behavior was already correct, only the assertion was stale.One gap found and left open, not guessed at:
F28E12is listed inTARGET_DEVICESbut has no entry in_DEVICE_PROFILES, sotest_cross_device.py::TestCompilationProfileCorrectnessstill fails for it (5 tests). This needs real hardware values (cross_compilerpath,target,target_c_mcpu,has_hard_npu) that aren't safe to fabricate — flagging it here rather than papering over it with placeholder values.CI trigger gap
tinyml-modelmakerdepends ontinyml-modelzoo(pip install -e tinyml-modelzooin theInstall dependenciesstep), but dependency-only changes totinyml-modelzoowouldn't trigger the test suite at all. Adds the missing path trigger.Note: this touches
.github/workflows/test-modelmaker.yml, which two of my other open PRs (pr/dataloader-pinning-fix,pr/mps-eval-fixes) also touch — each adds one path-trigger line at the same position. If merged after either, GitHub will show a trivial 3-line conflict; the correct resolution is just keeping all the added lines (a union), not choosing one over the other.🤖 Generated with Claude Code